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TECHNIQUE FOR DEFINING CONCEALMENT ORDER TO MINIMIZE ERROR PROPAGATION TECHNICAL FIELD 
This invention relates to a technique for establishing the order of concealment of macroblocks having lost and/or 
corrupted data. 

BACKGROUND ART In many instances, video streams undergo compression (coding) to facilitate storage and 
transmission. Not infrequently, such coded video streams incur data losses or become corrupted during transmission 
because of channel errors and/or network congestion. Upon decoding, the loss/corruption of data manifests itself as 
missing pixel values within one or more macroblocks within the decoded image. To reduce artifacts attributable to such 
missing/corrupted pixel values, a decoder will M conceai n such missing/corrupted pixel values by estimating the values 
from other macroblocks in the same image or from another image. The term conceal is a somewhat of a misnomer 
because the decoder does not actually hide missing or corrupted pixel values. 

Spatial concealment seeks to derive the missing/corrupted pixel values by using pixel values from other areas in the 
same image in reliance on the similarity between neighboring regions in the spatial domain. In contrast to spatial error 
concealment, temporal concealment attempts the recovery of the coded motion information, namely the reference 
picture indices and the motion vectors, to estimate the missing pixel values from at least one previously transmitted 
macroblock. When the errors do not affect isolated macroblocks but groups of contiguous macroblocks, concealing 
errors can require using information from already concealed blocks. 

The concealment of lost slices, as defined in the Main Profile of the ISO/ITU H. 264 standard for video compression 
exemplifies a typical situation requiring the use of information from previously concealed macroblocks. However such 
a strategy tends to propagate errors and compromises the quality of the restored image. 

One approach to overcoming this difficulty proposes to conceal entire columns of macroblocks, progressing inwards 
from the boundaries of the image towards the center. The macroblocks on the left and the macroblocks on the right 
undergo concealment independently of each other. Similarly, concealment within each column progresses 
independently up and down until the concealed macroblocks meet in the middle. This approach reduces error 
propagation by concealing the center of the image, which is typically difficult to predict but is visually important at the 
end of the process. However, accomplishing concealment in this manner reduces the ability to propagate meaninqful 
information from left to right and vice versa. 

Macroblocks on the left side of the image are predicted with no information coming from the right and vice versa Such 
behavior makes the propagation of diagonal contours across the missing region difficult. 

Thus, there is a need for a technique that accomplishes error concealment while overcoming the aforementioned 
drawback. 9 

BRIEF SUMMARY OF THE INVENTION Briefly, in accordance with the present principles, concealment of 
macroblocks in a rectangular array occurs by concealing the macroblocks having missing/corrupted values by 
progressing in the direction of one of the two larger sides of the array. Thus, for example, for a rectangular array 
having a greater width than height, concealment progresses by rows. 

Conversely, for a rectangular array that has a greater height than width, concealment progresses by columns. (For a 
square array, concealment can progress arbitrarily by rows or by columns.) The concealment of macroblocks by 
progressing in the direction of one of the two larger sides of the array facilitates the prediction of contours either in the 
vertical, horizontal or diaoonal direction, therebv imorovino the aualitv of concealment in contrast to Drior art 



concealment techniques that progressively conceal in a given direction irrespective of the dimensions of the array. 

BRIEF DESCRIPTION OF THE DRAWINGS FIGURE 1 illustrates an image partitioned into macroblocks; FIGURE 2 
illustrates the image of FIG. 1 after undergoing decoding when the image was previously coded and transmitted over 
an error-prone channel, with errors in the channel causing the loss of a rectangular array of macroblocks ; FIGURE 3 
illustrates the image of FIG. 2 during error concealment that progresses vertically by columns from the outer edges to 
the center of the image in accordance with the teachings of the prior art ; SUBSTITUTE SHEET (RULE 26) FIGURE 4 
illustrates the image of FIG. 2 during error concealment that progresses horizontally by rows in accordance with the 
present principles; and FIGURE 5 illustrates the image of FIG. 1 after undergoing decoding when the image was 
previously coded and transmitted over an error-prone channel, with errors in the channel causing the loss of a non- 
rectangular array of macroblocks ; and FIGURE 6 illustrates a partition of the non-rectangular array of lost 
macroblocks of FIG. 

5 into two rectangular sub-regions to enable error concealment in accordance with the present principles. 

DETAILED DESCRIPTION Block-based video compression techniques, such as embodied in the proposed ISO/ITU 
H. 264 video compression standard, operate by dividing a picture into slices, each slice comprising a set of 
macroblocks or macroblock pairs, with each macroblock coded in accordance with the standard. A macroblock 
typically comprises a squared region of 16x16 pixels. For coding purposes, macroblocks can be further partitioned into 
sub-macroblocks not necessarily squared. Each one of the sub-macroblocks can have different coding modes when 
the macroblock is encoded. FIGURE 1 depicts the partitioning of an original picture 100 into macroblocks 110. Note 
that the number of macroblocks within the picture 100 can vary depending on its size. 

In the illustrated embodiment, the picture 100 has a black stripe 120 that extends diagonally from the upper left-hand 
corner to the lower right-hand corner of the picture. The remaining portion of the picture 1 00 has a gray level for 
purposes of illustration. To facilitate transmission, the picture 100 will typically undergo coding, typically by any of 
several well- known block-based coding techniques, such as the technique described in the proposed ISO/ITU H. 264 
standard. During the transmission of the coded picture 100, one or more macroblocks 110 can become corrupted. In 
other words, the pixel values carried by such a macroblock 1 10 will become lost or tainted. An example of such a 
situation appears in FIG. 2 which illustrates the picture 100 following decoding. The decoded picture 100 in FIG. 2 has 
a number of rows of macroblocks with missing pixel values, all of which lie in a rectangular error region 130. For ease 
of illustration, the macroblocks 1 10 with missing pixel values appear in white in FIG. 2, as contrasted by those 
macroblocks with intact pixel values which appear either dark or gray, To assure optimal performance, a non- 
rectangular error region should be divided into rectangular sub-regions prior to undergoing concealment. Undertaking 
error concealment in accordance with the present principles on rectangular error regions, such as the error region 130 
of FIG. 4 assures a symmetrical progression from the top-most and the bottom-most row (equivalent^, from the left- 
most and the right-most columns). If the error region has larger width than height, division into rectangular sub-regions 
is done in order that each sub-region has the upper-most and the lower-most rows neighboring correctly received 
rows. Should the error region have a larger height than width, division into rectangular sub-regions is done in order 
that each sub-region has the left-most and the right-most rows neighboring correctly received columns. 

To better appreciate the process of partitioning a non-rectangular error region, refer to FIG. 5, which depicts the 
picture 100 of FIG. 1 after decoding when the image prior to decoding has undergone transmission through an error- 
prone channel causing resultant errors. Such errors manifest themselves in the picture 100 of FIG. 5 in the form of a 
non-rectangular error region 130'. In accordance with the present principles, the non-rectanguiar error region 
130'undergoes partitioning into two or more rectangular sub-regions, depicted as sub-regions 135 and 1352 in FIG. 5, 
with each sub-region having its upper and lower most rows bordering a correctly received row of macroblocks. 
Thereafter, the macroblocks in each of the sub-regions 1351 and 135z undergo concealment, by alternately scanning 
the rows of macroblocks in each of the sub-regions in opposite directions as depicted by the serpentine arrows within 
each sub-region. 

The foregoing describes a technique that establishes an order of concealment of macroblocks in a rectangular array 
that facilitates the prediction of contours either in the vertical, horizontal or diagonal direction, thereby improving the 
quality of concealment. 



